Disturbing Neighbors Diversity for Decision Forests

Ensemble methods take their output from a set of base predictors. The ensemble accuracy depends on two factors: the base classifiers accuracy and their diversity (how different are these base classifiers outputs from each other). An approach for increasing the diversity of the base classifiers is presented in this paper. The method builds some new features to be added to the base classifier training dataset. Those new features are computed (i) using the nearest neighbor instance from a very small previous randomly selected set and, (ii) the class this k-NN predicts for the instance. We tested this idea using decision trees as base classifiers. An experimental validation on 62 UCI datasets is provided for traditional ensemble methods, showing that ensemble accuracy and base classifiers diversity are usually improved.